1. Field of the Invention
The present invention relates to an information processing apparatus, an information processing system, and an information processing method to synthesize an image of a real environment with an image corresponding to the real environment to display.
2. Description of the Related Art
There is known a technique called augmented reality (AR) in which an image corresponding to a real environment is added to an image of the real environment.
In the AR, an image of a real environment is obtained by a camera or the like, and an additive image (hereinafter, additional image) corresponding to the real environment is synthesized with the obtained image of the real environment, to thereby be displayed on a display screen.
A user watches the image in which the image of the real environment and the additional image are synthesized, to thereby recognize as if an object or an image displayed as the additional image exists in the real environment.
In the AR, it is important to accurately establish a correspondence between an obtained image of a real environment and an additional image, that is, to match a position, size, angle, and the like of the additional image with the real environment. In a case where the correspondence is not established accurately, the position, size, angle, and the like of the displayed additional image with respect to the real environment becomes unnatural.
For example, Hirokazu Kato, Mark Billinghurst, Koichi Asano, and Keihachiro Tachibana, “An Augmented Reality System and its Calibration based on Marker Tracking”, Journal of The Virtual Reality Society of Japan, Vol. 4, No. 4, 1999, http://intron.kz.tsukuba.ac.jp/tvrsj/4.4/kato/p-99_VRSJ4—4.pdf (hereinafter, referred to as Non-patent Document 1) describes an AR system using a marker. In this system, image information on a marker (colored square having a predetermined size) is previously registered. In a case where an image of a real environment includes the image of the marker, the marker is detected in the image of the real environment, and a spatial positional relationship of a camera to the marker is calculated based on information on the size, angle, and the like. Based on the positional relationship, a virtual object is displayed on an HDM whose relative position to the camera is fixed, and a user is capable of watching the virtual object together with the real environment.
Further, the AR is capable of, not only displaying the real environment in relation with the additional image, but also changing the additional image according to an input by a user. For example, when a user aims a pointer at a real environment while watching the real environment to which the additional image is added, an operation of changing the additional image corresponding to the instructed position of the real environment is enabled. Japanese Patent Application Laid-open No. 2007-75213, paragraph 0086, FIG. 7 (hereinafter, referred to as Patent Document 1) describes an information processing program enabling such an operation.
Patent Document 1 describes an apparatus including two markers (infrared ray light sources) provided on the periphery of a monitor so as to be spaced apart, and a controller having an image pickup device capable of picking up an image of those markers. When a user operates the controller while aiming the controller at the monitor, a spatial positional relationship of the controller to the monitor is calculated based on the distance, angle, and the like of the two markers picked up by the image pickup device, and the operation by the user is input to the apparatus.